Overview: Enterprise Streaming Discovery
A single, optimized job type included in eCapture is named Enterprise Streaming Discovery. Several characteristics make Streaming Discovery unique:
- It combines both a traditional Discovery Job and a Data Extract Job.
- It uses its method to push data through to the review process and reduces the number of starts and stops required by traditional methods.
- Images may be generated during a Streaming Discovery Job allowing for automatic loading into a review platform such as Ipro Eclipse.
- Document families are available for the review process sooner due to the use of family-based task distribution. All document families keep moving forward through the Streaming Discovery phases as soon as they are ready. The Enterprise Streaming Workers are constantly processing data so the data can move through the filtering phase and then to the export phase.
Streaming Discovery is particularly useful for large data sets. For example, portions from a large data set may have just finished the Discovery phase, while other portions from this same large data set were already pushed to the review tool. Unlike the traditional method, there is no need to wait until the entire data set completes a single phase before it can move on to the next phase. This former method "chunked" the data, thereby requiring stops and starts at each phase and delaying the review process.
The Enterprise Streaming Worker, used for Streaming Discovery, is built into eCapture’s existing framework. Streaming Worker uses in-memory processing, multithreading, and newer technology to take advantage of the powerful computing capability built into newer, high-end computers. At the same time, Streaming Worker can run on existing hardware, including virtual computers.
Streaming Discovery Workflow
The Streaming Discovery Workflow is as follows:
- After a data set (mailstores, PSTs, etc.) is loaded , its files are extracted to disk and distributed to the Streaming Workers.
- The files then enter the Discovery phase, which encompasses both discovery and data extraction.
- The completed document families are retrieved and sent through the basic filtering phase before they are queued for exporting.
- Exports are driven by the existing eCapture Export Series method. Data may be pushed to Eclipse, Relativity, and to disk if needed. An Export Interval value is set to determine how often a Streaming Discovery Job will pull document families for Export.
- The following diagram demonstrates the workflow process for Streaming Discovery.
Streaming Discovery Job Right-Click Menu Options
When you right-click a Streaming Discovery Job, a context menu displays showing the following options:
- New Streaming Discovery Job
- Rename Streaming Discovery Job
- Export Streaming Discovery Job – this option is used to manually export a completed Streaming Discovery Job to Disk or to the Review application. This option is enabled only if an Export Series was not selected/created on the Export Options tab for the Streaming Discovery Job. During the Streaming Discovery Job, you normally give export intervals. The re-export takes all exported items and combines them into a single item for the wizard.
- Delete Streaming Discovery Job
- Reporting – displays a sub menu that contains a selection of different report types. For more information about the report types, seeOverview: Reports.
Convert Legacy Cases to Streaming Discovery Cases
eCapture allows you to convert a legacy case (which includes Standard Discovery, Data Extract, and Process jobs) to a Streaming Discovery case, which provides greater efficiency and improved performance when running jobs. You can launch this conversion process by using the Case Hash Conversion option, available from a right-click context menu in the eCapture Controller. This option rehashes previously imported emails and stores the generated hash values and item IDs in a new table, allowing Streaming de-duplication. For more information about this task, see Convert Legacy Cases to Streaming Discovery Cases.
Related Topics
Overview: Enterprise Integration